As processor speeds continue to increase, memories providing data to the processor have become more and more of a bottleneck. In an effort to speed memory access, high speed caches were created to deliver data to processors. Generally, a cache only stores a fraction of the data stored in main memory. A cache “hit” occurs when the cache contains data the processor is requesting. A cache “miss” occurs when the cache does not contain data the processor is requesting. When a cache miss occurs, the data must be retrieved from main memory or disk. The time to fetch the data when a cache miss occurs, even from main memory, can be much greater than when a cache hit occurs. Increasing the percentage of cache hits and decreasing the number of cache misses, therefore, increases the overall performance of a computer system.
One approach to increasing performance is through the use of optimizing compilers. While for certain kinds of optimizations, this works well, for other kinds of optimizations it provides little if any benefit. Optimizing compilers are typically poor, for example, at rewriting an inefficient algorithm.
Another approach to increasing performance is to profile an executing program. Profiling generates data about how long a program spends in each subroutine. Profiling tools may tell a programmer the percentages of time consumed in each subroutine of an executing program, but they fail to tell the programmer why the program spends the time it does in each subroutine. A programmer may unwittingly make changes that decrease program performance.